Papers with NLP evaluation practices
Human-Centered Evaluation of Language Technologies (2024.emnlp-tutorials)
Copied to clipboard
| Challenge: | a lack of human-centered considerations about people’s needs for language technologies is causing an “evaluation crisis” in NLP. |
| Approach: | This tutorial introduces perspectives and methodologies from human-computer interaction (HCI) it will introduce what to evaluate for, how generalizable the results are to the real-world contexts, and pragmatic costs to conduct the evaluation. |
| Outcome: | This tutorial introduces perspectives and methodologies from human-computer interaction (HCI) the tutorial will also encourage reflection on how these HCI perspectives and methods can complement NLP evaluation through Q&A discussions and a hands-on exercise. |